1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing system, an imaging apparatus, and an image processing method, which are used for correcting distortion of a captured image. More particularly, the present invention relates to an image processing apparatus, an image processing system, an imaging apparatus, and an image processing method, which are capable of performing distortion correction on an image, which is taken by using a converging lens group having an optical zoom mechanism, according to a zoom position.
2. Description of the Related Art
Hitherto, it has been known that in imaging apparatuses, such as a video camera and a still camera, distortion occurs in a captured image under the influence of distortion and aberration characteristics of an imaging lens. In a case where a high-precision and high-performance lens, such distortion is inconspicuous. However, in a case where a low-cost lens is used, or where an optical zoom lens is used, it is difficult to circumvent the influence of distortion on image quality. Accordingly, in recent years, there has been proposed an image processing system adapted to correct such optical distortion by signal processing.
FIG. 27 is a block diagram showing an example of the configuration of an image processing system of related, art having such a signal processing function.
The image processing system shown in FIG. 27 has an imaging apparatus 3 and a preprocessor 4. Incidentally, it is assumed herein that a digital video camera is provided as the imaging apparatus 3. Further, the preprocessor 4 is configured to be, for example, a personal computer provided on the exterior of this imaging apparatus 3.
The imaging apparatus 3 has an optical block 101, an imaging element 102, an image preprocessor 103, an image signal processor 104, an image memory 105, a display processor 106, a monitor 107, a compression/decompression processor 108, a recording/reproducing section 109, recording media 110, a control microcomputer (hereunder sometimes referred to simply as a microcomputer) 111, a correction parameter decoder 312, and a distortion correction memory 313. Further, the preprocessor 4 has a correction parameter deriving section 401 and a correction parameter encoder 402.
In the imaging apparatus 3, reflection light from an object to be imaged is incident upon the optical block 101. The optical block 101 has plural lenses, a drive mechanism therefor, and so on. The incident light is converged to the imaging element 102. The imaging element 102 includes CCDs (Charge Coupled Devices) and so forth. The incident light is converted into electrical signals that are supplied to the image preprocessor 103. In the image preprocessor 103, CDS (Correlated Double Sampling), AGC (Auto Gain Control), A/D conversion and the like are performed on image signals outputted from the imaging element 102. The digitalized image signals are supplied to the image signal processor 104.
In the image signal processor 104, the inputted digital image signal is stored in the image memory 105. Further, an image quality correction process, such as a distortion correction process, according to correction amount parameters received from the correction parameter decoder 312 is performed on this digital image signal. The processed image signal is supplied to the display processor 106. Then, an image signal to be displayed on the monitor 107 is generated therein. Consequently, the captured image is displayed in the monitor 107. Incidentally, the monitor 107 may be constituted by, for example, a LCD (Liquid Crystal Display).
Further, the image signal having undergone the image quality correction process in the image signal processor 104 is subjected to a compression/decompression process in a predetermined image format in the compression/decompression processor 108. Then, a resultant signal is written to the recording media 110 by the recording/reproducing section 109. Accordingly, the recording of the captured image is performed. Incidentally, for instance, magnetic tape, a semiconductor memory, an optical disk, and a hard disk may be used as the recording media 110.
On the other hand, in a case where image data recorded in the recording media 110 is reproduced, this image data is read by the recording/reproducing section 109 and then undergoes a decompressing-decoding process in the compression/decompression processor 108. The processed image data signal is supplied to the display processor 106, whereby a reproduced image is displayed in the monitor 107.
Such a recording/reproducing operation of an image is controlled by the control microcomputer 111. The control microcomputer 111 outputs a command or the like for instructing the image signal processor 104 to perform a predetermined operation, to the image signal processor 104 in response to a control signal received from a user interface (I/F) (not shown). Moreover, the control microcomputer 111 supplies information on the position of a lens in the optical block 101 to the correction parameter decoder 312.
The correction parameter decoder 312 performs decompression (decoding) on compressed data, which is read from the distortion correction memory 313, according to the information and the like, which is supplied from the control microcomputer 111, to thereby obtain a correction amount parameter corresponding to each of pixels. Then, the correction parameter decoder 312 supplies the correction amount parameters to the image signal processor 104. The distortion correction memory 313 supplies a distortion correction parameter held therein to the correction parameter decoder 312 in response to a request therefrom.
Hereinafter, an example of the distortion correction process to be performed in the image signal processor 104 is described. In the image signal processor 104, for example, a distortion-corrected image is divided into regions in a lattice form. Then, an interpolation calculation corresponding to each direction is performed by using coordinates of plural lattice points in the x-direction and the y-direction of the region including a pixel undergoing correction. Accordingly, distortion is corrected.
The correction parameter decoder 312 outputs correction amount parameters providing, for example, correction coordinates and interpolation phases in the X-direction and the y-direction. Further, the image signal processor 104 calculates correction vectors and interpolation coefficients in the respective directions. Then, the image signal processor 104 serially performs one-dimensional interpolation calculations respectively corresponding to the directions. Distortion in each of the directions can be corrected by performing such a relatively low load process.
Meanwhile, in the preprocessor 4, the correction parameter deriving section 401 generates the distortion correction coordinates of all pixels of the captured image according to lens data of lenses mounted in the optical block 101, and outputs the generated coordinates to the correction parameter encoder 402.
The correction parameter encoder 402 partitions the distortion-corrected image in a lattice form, and compresses the distortion correction coordinates of all pixels, which are outputted from the correction parameter deriving section 401, in the x-direction and the y-direction, respectively, by utilizing the lattice positions thereof. Consequently, the generated distortion correction parameters are stored in the distortion correction memory 313.
Incidentally, the distortion correction parameters may be generated in the correction parameter encoder 402 in real time every time when the distortion correction process is performed, and also may be supplied to the distortion correction memory 313. Alternatively, the distortion correction parameters may collectively be loaded into the distortion correction memory 313 in an initial operation, such as a power-on operation.
The correction parameter decoder 312 receives the designation of coordinates, which are caused to undergo the correction, from the image signal processor 104, and reads the distortion correction parameters, which correspond to the region of the lattice, from the distortion correction memory 313, and performs the interpolation calculation in the x-direction and the y-direction thereby to decompress the distortion correction coordinates and to generate distortion correction parameters.
Accordingly, preliminarily, the entire image is divided into the regions of the lattice, and the distortion correction coordinates of each of the pixels are compressed. The distortion correction coordinates are decompressed in the correction parameter decoder 312. With this configuration, an amount of necessary data and a load to the calculation process are reduced, so that the process can be performed in real time.
Meanwhile, it is known that distortion characteristics largely change when the position of the lens changes in the imaging apparatus, which has an optical zoom function, by performing a zoom operation. Therefore, it is preferable to utilize optimal distortion correction ordinates according to the change in the position of the lens.
In the imaging system described above, for example, the correction parameter encoder 402 generates the distortion correction parameter associated with each of lens positions and stores the generated distortion correction parameters in the distortion correction memory 313. Additionally, the control microcomputer 111 informs the correction parameter decoder 312 of the lens position. The correction parameter decoder 312 selectively decodes the distortion correction parameter associated with this lens position. Accordingly, appropriate distortion correction according to the lens position can be performed.
Further, FIG. 28 is an example of a graph showing the relation between the lens position and a distortion amount during a zoom operation.
As shown in FIG. 28, a distortion amount-lens position characteristic is nonlinear. Moreover, change amounts at points in the image differ from one another. Accordingly, it is necessary for correcting distortion to prepare distortion correction data for all the pixels, which corresponds to each of all the lens positions. A huge calculation amount is needed for calculating these data. Further, mass memory capacity is needed for holding these data.
Incidentally, a solid-state imaging camera configured to correct distortion of an image by reading image data from a solid-state image sensing device according to geometrical deformation in a case where the imaging position of an imaging zoom lens is within a position, in which a large distortion aberration occurs, has been provided an example of a conventional apparatus enabled to perform distortion correction according to the lens position by using a low-cost circuit (for example, see Paragraph Nos. [0015] to [0020], and FIG. 1 in the specification of Japanese Patent No. 2925871, which will be referred to as Patent Document 1 hereinafter).